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DETAILED ACTION 
Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including tlie fee set 
fortli in 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on March 
12, 2010 has been entered. 

Claim Rejections - 35 USC § 101 

2. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or 
composition of matter, or any new and useful improvement thereof, may obtain a patent 
therefor, subject to the conditions and requirements of this title. 

The USPTO "Interim Guidelines for Examination of Patent Applications for Patent 
Subject Matter Eligibility" (Official Gazette notice of 22 November 2005), Annex IV, 
reads as follows: 

In contrast, a claimed computer-readable medium encoded with a computer program is a 
computer element which defines structural and functional interrelationships between the 
computer program and the rest of the computer which permit the computer program's 
functionality to be realized, and is thus statutory. See Lowry, 32 F.3d at 1583-84, 32 
USPQ2d at 1035. 
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Claims that recite nothing but the physical characteristics of a form of energy, such as a 
frequency, voltage, or the strength of a magnetic field, define energy or magnetism, per 
se, and as such are nonstatutory natural phenomena. O'Reilly, 56 U.S. (15 How.) at 112- 
14. Moreover, it does not appear that a claim reciting a signal encoded with functional 
descriptive material falls within any of the categories of patentable subject matter set forth 
in Sec. 101. 

... a signal does not fall within one of the four statutory classes of Sec. 101. 

... signal claims are ineligible for patent protection because they do not fail within any of 
the four statutory classes of Sec. 101 . 

Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter as follows. Claim 9 is drawn to functional 
descriptive material recorded on a computer readable medium. However, the broadest 
reasonable interpretation of a claim drawn to a computer readable medium (also called 
machine readable medium and other such variations) typically covers forms of non- 
transitory tangible media and transitory propagating signals per se in view of the 
ordinary and customary meaning of computer readable media. When the broadest 
reasonable interpretation of claim covers a signal perse, the claim must be rejected 
under 35 U.S.C. § 101 as covering non-statutory subject matter. See In re Nuijten, 500 
F.3d 1346, 1356-57 (Fed. Cir, 2007) (transitory embodiments are not directed to 
statutory subject matter) and Interim Examination Instructions for Evaluating Subject 
Matter Eligibility Under 35 U.S.C. § 101, August 24, 2009; P. 2. 

Because the full scope of the claim as properly read in light of the disclosure 
encompasses non-statutory subject matter, the claim as a whole is non-statutory. The 
examiner suggests amending the claim as "A program embodied on a non-transitory 
computer readable medium" to include the disclosed non-transitory tangible computer 
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readable media, while at the same time excluding the intangible media such as signals, 
carrier waves, etc. 



Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 1 02 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious 
at the time the invention was made to a person having ordinary sl<ill in the art to which said 
subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

4. Claims 1,2, 5-7, 9, 11, 12, 14, and 17 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over the combination of Mitsui (US 7,222,235 B1), Barr et al. (US 
2004/0039914 A1), and Rhoads (US 6,411,725 81). 

Regarding claim 1, Mitsui discloses an information attaching/detecting system 
comprising: 

- object extracting means, for extracting a plurality of regions, from within the 
image (as depicted in figure 15, numeral 106 will perform the object extracting means, 
"[t]he image inputting apparatus 101(m) inputs an image and converts the input image 
to electronic data. The image inputting apparatus 101(m) may be a scanner or a digital 
camera. The image inputting apparatus 101(m) outputs the input image data as an 
original image A(m). The image processing apparatus 106(m) has a function of inputting 
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the original image A(m) from tlie image inputting apparatus 101(m) and outputting a 
processed image C(m). The image processing apparatus 106(m) generates the 
processed image C(m) by embedding a digital watermark into the original image A(m)" 
at column 22, line 1 1 . The plurality of photographed objects is shown in figure 33, "FIG. 
33 shows how the digital watermark is embedded in the form Y ... the digital watermark 
is shaded for an explanation. For example, different kinds of digital watermarks are 
embedded for each of the region SI and the region S2 because the character size is 
different between the region SI and the region S2 ... Furthermore, because the region 
S3 is a photograph, the digital watermark, which is different from in the region S1 and 
the region S2, is embedded in the region S3" at column 38, line 56. See also figure 32 
which indicates a plurality of regions, such as character with REGION 1 and REGI0N2 
and photograph with REGION 3 and REGION n, is valid for object extraction); 

- information attaching means for attaching different information to each of the 
regions that do not overlap with other regions and acquiring said information-attached 
image (as discussed above and shown in figure 15, numeral 106 will perform the 
information attaching means. Different information attaches to each not overlap regions 
is shown in figure 33, "FIG. 33 shows how the digital watermark is embedded in the 
form Y ... the digital watermark is shaded for an explanation. For example, different 
kinds of digital watermarks are embedded for each of the region S1 and the region S2 
because the character size is different between the region SI and the region S2 ... 
Furthermore, because the region S3 is a photograph, the digital watermark, which is 
different from in the region SI and the region S2, is embedded in the region S3" at 
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column 38, line 56. See also figure 32 which indicates a plurality of regions, such as 
character with REGION 1 and REGI0N2 and photograph with REGION 3 and REGION 
n, is valid for embedding information); 

- detection means for detecting said information from received data for each of 
said plurality regions contained in said information-attached image (as depicted in figure 
15, numeral 108 will detect the embedded information received from transmitted 
information-attached image. Figure 19 shows a block diagram for the information 
detection. Detailed discussion about the detection processing can be found from 
column 24, line 22 to column 25, line 8). 

Mitsui discloses that regions are used for information attaching/detecting. Mitsui 
does not explicitly disclose that the regions are persons facial regions. Furthermore, 
Mitsui discloses the detection means from the transmitted information-attached image. 
Mitsui does not explicitly disclose that the received data can be a photographed-image 
by photographing an image reproducing medium on which information-attached image 
is reproduced. 

Barr, in the field of endeavor of security in watermarking ("for authenticating 
electronic or physical media objects using digital watermarks" in paragraph 9, line 1), 
teaches the "application of digital watermarking is for the authentication of physical and 
electronic media objects, like images, video, audio, and printed media" in paragraph 6, 
line 1 . "Another way to authenticate the object is to embed information about the object 
or the bearer of the object (e.g., in photo ID or other secure documents)" in paragraph 7, 
line 1 . "One aspect of the invention is a method for creating an identification document 
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having a biometric image. Tlnis metliod identifies a feature location in the biometric 
image, generates a digital watermark signal, and embeds the digital watermark signal in 
the biometric image such that the digital watermark location is dependent on the feature 
location" in paragraph 13, line 1 . As shown in figure 3, a person facial photo is used for 
document ID information embedding, "In the case of a photo identification document 
that includes a photo of the bearer's face, added security can be attained by including 
facial biometric information in the digital watermark, which is then embedded in the 
facial photo" in paragraph 70, line 7. 

It would have been obvious at the time the invention was made, that one of 
ordinary skill in the art would have been motivated to include the information 
attaching/detecting system Mitsui made with the application of embedding information 
on person facial region as taught by Barr, in order to enhance the security as stated 
above by Barr. 

The combination of Mitsui and Barr does not explicitly disclose that the received 
data can be a photographed-image by photographing an image reproducing medium on 
which information-attached image is reproduced. 

Rhoads, in the field of endeavor of multimedia signal processing ("in particular 
relates to encoding information into and decoding information from video objects" at 
column 1 , line 30), teaches a robust data embedding methodology as shown in figure 2, 
"[i]n FIG. 2, physical objects 200 are pre-watermarked in a manner that survives the 
video capture process 202. For an example of a watermarking process that survives 
digital to analog conversion (e.g., printing a digital image on a physical object), and then 
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analog to digital conversion (e.g., capture via a video camera) ... The resulting video is 
then transmitted or broadcast 204" at column 7, line 29. Thus, the watermark can 
survive by printing this watermark embedded digital image and then capture this printed 
image via camera. Without departing from the scope and spirit of Rhoads' 
methodology, a second Image captured by the camera can be an image from printing 
which created by Mitsui information attached first image. 

It would have been obvious at the time the invention was made, that one of 
ordinary skill in the art would have been motivated to enhance the information 
attaching/detecting system of the Mitsui and Barr combination, with the application of 
data embedding methodology as taught by Rhoads, so the embedded data can survive 
various scenarios as discussed above. 

Regarding claim 2, information attaching means is means for acquiring said 
information-attached image by hiddenly embedding said information in said image ("An 
image processing system can be provided such that the image providing apparatus 
recognizes a format of an image file, and provides the image file in which the digital 
watermark is embedded in a part of a region based on the format ..." at Mitsui column 5, 
line 61. See also Rhoads figure 4, numeral 404 "...watermark encoding process 404 
encodes the auxiliary information into the content. ..." at Rhoads column 8, line 13). 
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Regarding claim 5, image pick-up means is a camera provided in a portable 
terminal (discussed in claim 1 by Mitsui, digital camera can be the image pick-up 
means). 

Regarding claim 6, said information is location information representing storage 
locations of audio data correlated with said plurality of photographed objects ("Another 
aspect of the invention is a method for using a watermark that has been encoded into a 
video signal ... The watermark may include a direct (e.g., URL or network address) or 
indirect link (e.g., object identifier) to the web site ..." at Rhoads column 2, line 35), 
and which further comprises audio data acquisition means for acquiring said audio data, 
based on said location information (as depicted in Rhoads figure 1 A, numeral 1 14: "a 
user interface 1 14 executes and provides visual, audio, or audio-visual information to 
the user ... user interface receives input from the user, selecting a video object. In 
response, it performs an action associated with the selected object using the auxiliary 
object information decoded from the watermark ..." at Rhoads column 4, line 17. See 
also "The watermark may carry information or programmatic action. It may also link to 
external information or an action, such as retrieval and output of information stored 
elsewhere in a database, website, etc. Watermark linking enables the action associated 
with the watermark to be dynamic ..." at Rhoads column 4, line 29). 

Regarding claim 1 1 , the information device according to claim 1 , further 
comprising: 
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- a mobile communication terminal having a camera (as discussed in claim 1 and 
suggested by Rhoads a camera for the mobile communication terminal); 

- a server (as depicted in Rhoads figure 8, numeral 802 is a server); 

- wherein the camera of the mobile terminal photographs the image reproducing 
medium (as discussed in claim 1 and suggested by Rhoads a camera of the mobile 
communication terminal photographs the image reproducing medium); 

- wherein the input means and the detecting means are disposed in the server 
(as depicted in Rhoads figure 8, the server #802 receives input and parses and extracts 
to detect information: "The server, in response to receiving the message (828), parses it 
and extracts an index used to look up a corresponding action in a database (830) that 
associates many such indices to corresponding actions ..." at Rhoads column 14, line 
44). 

Regarding claim 12: - wherein the information is a location of audio data and 
the server acquires said audio data using said location and transmits said audio data to 
said mobile communication terminal (the combination of Mitsui and Rhoads teaches the 
mobile terminal about the location: "watermark may include a direct (e.g., URL or 
network address) or indirect link (e.g., object identifier) to the web site ..." at Rhoads 
column 2, line 44, to get audio data: "user interface 114 (Rhoads figure 1A) executes 
and provides visual, audio, or audio-visual information to the user ... user interface 
receives input from the user, selecting a video object. In response, it performs an action 
associated with the selected object ..." at column 4, line 17. Furthermore, "The server. 
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in response to receiving the message (828) (Rhoads figured), parses it and extracts an 
index used to \ook up a corresponding action in a database (830) that associates many 
such indices to corresponding actions ..." at Rhoads column 14, line 44). 

Regarding claim 14: - a region setting section, for setting block regions 
corresponding to each photographed object that do not overlap with other regions that 
include other photographed objects (as discussed in claim 1 , the figure 33 of Mitsui 
contains plurality of not overlapped regions with photographed objects such as SI , S2, 
S2, and S4); 

- an information attaching section, for attaching different information to each of 
the plurality of block regions which are set by the region setting means (as discussed in 
claim 1, the information attaching means for, information attached to each region is 
different). 

Regarding claims 7 and 9: - extracting a plurality of facial regions of persons from 
within the image (discussed in claim 1, object extracting means for); 

- attaching different information to each of the plurality of facial regions in said 
image that respectively and acquiring said Information-attached image (as discussed in 
claim 1 for information attaching means for. See also Mitsui figure 33, "FIG. 33 shows 
how the digital watermark is embedded in the form Y ... the digital watermark is shaded 
for an explanation. For example, different kinds of digital watermarks are embedded for 
each of the region SI and the region S2 because the character size is different between 
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the region SI and tlie region S2 ... Furthermore, because the region S3 is a 
photograph, the digital watermark, which is different from in the region SI and the 
region S2, is embedded in the region S3" at Mitsui column 38, line 56); 

- receiving photographed-image data obtained by photographing an image 
reproducing medium, on which the information-attached image is reproduced, with 
image pick-up means (as discussed in claim 1, input means for); 

- detecting said information from said photographed-image data for each of said 
plurality of facial regions contained in said information-attached image (as discussed in 
claimi, detection means for); 

- outputting the information corresponding to each of the plurality of facial 
regions, based on the detected information ("Wherever the video is distributed, a 
decoding process may be inserted to decode information about the video object from a 
watermark embedded in the video signal. This information may then be used to trigger 
an action, such as fetching graphics and displaying it to the user ..." at Rhoads column 
4, line 61); 

- wherein the information attached related to a person corresponding to the one 

of the facial regions to which the information is attached (discussed in claim 1 by Barr 
for person facial region with embedded information). 

Regarding claim 17, wherein the outputting comprises providing a visual or audio 
display via at least one of a audio device and a display monitor (as discussed in claim 7 
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for outputting detected information "may then be used to trigger an action, sucli as 
fetcliing grapliics and displaying it to the user ..." at Rhoads column 4, line 64). 

5. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Mitsui, Barr, and Rhoads as applied to claim 1 discussed above, and 
further in view of Motta et al. (US 6,726,103 B1). 

Regarding claim 4, the Mitsui, Barr, and Rhoads combination discloses an 
information attaching/detecting system with image pick-up means to receive 
photographed-image data. 

The Mitsui, Barr, and Rhoads combination does not explicitly disclose a way to 
correct the geometrical distorted photographed-image data. 

Motta, in the same field of endeavor of digital imaging ("relates to the field of 
image sensors and imaging systems" at column 1 , line 8), teaches a geometric 
correction processing system for the photographed-image data as depicted in figure 3. 
"Data processing may include but is not limited to data interpolation, noise reduction, 
color adjustment, and/or geometric corrections due to optical aberrations ..." at column 
4, line 4. 

It would have been obvious at the time the invention was made, that one of 
ordinary sl<ill in the art would have been motivated to provide the information 
attaching/detecting system made by Mitsui, Barr, and Rhoads combination with the 
geometrical distortion correction processor as taught by Motta, in order to provide "[a] 
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system that can self test, detect errors, and correct such errors ... and would be of great 
value to many businesses and industries" at Motta column 2, line 29. 

6. Claims 15 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over the combination of Mitsui, Barr, and Rhoads as applied to claim 1 discussed 
above, and further in view of Hsieh et al. (US 7,003,135 B2). 

Regarding claims 15 and 16, the Mitsui, Barr, and Rhoads combination discloses 
an information attaching/detecting system to extract a plurality of facial regions of 
persons from within the image. 

The Mitsui, Barr, and Rhoads combination does not explicitly disclose skin 
color/face contours can be used for facial region extraction. 

Hsieh, in the field of endeavor of faces tracking ("which is able to effectively 
detect and track a plurality effaces in real-time" at column 2, line 7), based on the fact 
that "[i]n order to efficiently separate the face regions from the complex backgrounds, a 
preferable approach is to use the hybrid information of color and motion. Then, the 
segmented regions are further verified to determine whether the correct faces are 
extracted from the remaining distracters" at column 1 , line 42, teaches a methodology to 
track no-overlap human faces as shown in figure 1 , numeral 10 and overlapped faces 
with: "a face-like region generator having a skin region extractor, a motion analyzer and 
a silhouette analyzer. The skin region extractor generates a plurality of skin regions by 
detecting skin color pixels of an input image. The motion analyzer determines possible 
face-like regions from the skin regions based on moving information of the input image. 
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The silhouette analyzer analyzes whether there exists a raised shape in the image so 
as to separate connected regions ..." at column 2, line 39. Thus, Hsieh discloses a 
methodology to detect regions which are person facial regions for real-time tracking by: 

a) Skin region extractor for facial regions to generate skin-color regions by detecting 
skin color pixels of an input image (claim 15). 

b) Silhouette analyzer for facial regions to generate face contours (claim 1 6). 
It would have been obvious at the time the invention was made, that one of 

ordinary skill in the art would have been motivated to include the information 
attaching/detecting system made by Mitsui, Barr, and Rhoads combination with the 
application of face-like region generator as taught by Hsieh, in order to effectively 
tracking human faces in real-time as stated above by Hsieh. 



Response to Arguments 

A) Summary of Applicant's Remark : 

"Thus, even if Mitsui could be combined with Hsieh and Rhoads as suggested by 
the Examiner, the applied combination fails to disclose "wherein the information 
attached relates to a person corresponding to the one of the facial regions to which the 
information is attached," as recited in claim 1 ." at Remarks page 9, line 9. 
Examiner's Response : 
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Applicant's argument is moot in view of the new grounds of rejection advanced 
herein above. Specifically, the Barr et al. (US 2004/0039914 A1) reference now 
teaches the concept of embedding information on person facial region. Refer to the 
rejections above for further discussion. 



Conclusion 

7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Eueng-nan Yeh whose telephone number is 571-270- 
1586. The examiner can normally be reached on Monday-Friday 8AM-4:30PM EDT. 

8. If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on 571-272-7415. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
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you have questions on access to tine Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Eueng-nan Yeh 
Assistant Patent Examiner 
Art Unit: 2624 
/E.Y./ 

A/IKKRAM BALI/ 

Supervisory Patent Examiner, Art Unit 2624 



